Regions-of-interest (ROI) scalability is an essential feature for some scalable video coding applications. Users may want to have different spatial/temporal/quality scalability inside and outside of ROIs.
Flexible macroblock ordering (FMO) has been discussed to implement the feature of ROI. The International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) standard/International Telecommunication Union, Telecommunication Sector (ITU-T) H.264 standard (hereinafter the “MPEG4/H.264 standard” or simply the “H.264 standard”) requires that each macroblock in an image be included in a slice group, and that all slice groups be coded in primary coded pictures (although it is not required for redundant pictures). The H.264 standard does not allow any missing slice groups, which means that even for slice groups that include non-ROIs, we still need to code these slice groups and send them into the network. It may be argued that the encoder has the option to control the bits for the macroblocks outside the ROI. For example, in one prior art implementation, the encoder can code all the macroblocks in non-ROIs to be BL_SKIP or INTRA_BL mode. Even though the bits required to signal the BL_SKIP or INTRA_BL mode might be small, it still takes the encoder extra effort to code them. More importantly, to code non-ROIs requires extra NAL units and, hence, extra bit rate overhead. For example, in the cases of a ROI versus non-ROI split of (¼ versus ¾) and (½ versus ½) for a quarter common intermediate format (QCIF) base layer and common intermediate format (CIF) enhancement layer (30 fps), the bit rate overhead to transmit INTRA_BL mode for the non-ROI region in the enhancement layer over real-time protocol (RTP) is approximately calculated as 12 kbps and 9 kbps, respectively. At the same time it increases the burden on the router to parse all these NAL units. In addition, the decoder still needs to parse those bit streams for non-ROIs. It should be noted that in the Joint Scalable Video Model (JSVM) Version 3.0, supporting INTRA_BL mode for all the macroblocks in the enhancement layer regardless of base layer macroblock types requires multi-loop decoding, which may not be allowed for some profiles. This constraint limits the usage of INTRA_BL mode if the encoder only supports single loop decoding. Thus, currently, JSVM Version 3.0 does not allow the use of INTRA_BL mode for all MBs in non-ROI regardless of single or multiple decoding loop.